HARD: Hybrid Adaptive Resource Discovery for Jungle Computing

In recent years, Jungle Computing has emerged as a distributed computing paradigm based on simultaneous combination of various hierarchical and distributed computing environments which are composed by large number of heterogeneous resources. In such a computing environment, the resources and the underlying computation and communication infrastructures are highly-hierarchical and heterogeneous. This creates a lot of difficulty and complexity for finding the proper resources in a precise way in order to run a particular job on the system efficiently. This paper proposes Hybrid Adaptive Resource Discovery (HARD), a novel efficient and highly scalable resource-discovery approach which is built upon a virtual hierarchical overlay based on self-organization and self-adaptation of processing resources in the system, where the computing resources are organized into distributed hierarchies according to a proposed hierarchical multi-layered resource description model. The proposed approach supports distributed query processing within and across hierarchical layers by deploying various distributed resource discovery services and functionalities in the system which are implemented using different adapted algorithms and mechanisms in each level of hierarchy. The proposed approach addresses the requirements for resource discovery in Jungle Computing environments such as high-hierarchy, high-heterogeneity, high-scalability and dynamicity. Simulation results show significant scalability and efficiency of the proposed approach over highly heterogeneous, hierarchical and dynamic computing environments

A Self-organizing and Self-configuration Algorithm for Resource Management in Service-oriented Systems

With the ever increasing deployment of service-oriented distributed systems in large-scale and heterogeneous computing environments, clustering and communication overlay topology design has become more and more important to address several challenging issues and conflicting requirements, such as efficient scheduling and distribution of services among computing resources, reducing communication cost between services, high performance service and resource discovery while considering both inter-service and inter-node properties and also increasing the load distribution and the load balance. In this paper, a four-stage hierarchical clustering algorithm is proposed which automates the process of the optimally composing communicating groups in a dynamic way while preserving the proximity of the nodes. The simulation results show the performance of the algorithm with respect to load balance, scalability and efficiency

A Specification-based Anycast Scheme for Scalable Resource Discovery in Distributed Systems

Anycast is a powerful paradigm for managing and locating resources in large scale distributed computing systems. This paper presents a novel specification-based anycasting scheme for resource discovery in such environments. The effectiveness of our proposal is demonstrated through simulation results, in which we observed a remarkable performance enhancement in different aspects (such as discovery latency, discovery cost, discovery load, etc.) over similar non-anycast based discovery methods

Very large scale high performance computing and instrument management for high availability systems through the use of virtualization at the Square Kilometre Array (SKA) telescope

The Square Kilometer Array (SKA) Telescope, is an ongoing project set to start its building phase in 2018 and be ready for first light in 2020. The first part of the project, the SKA1 will be comprised of 130.000 low frequency antennas (50 MHz to 350 MHz) and 200 mid frequency antennas (350 MHz to 15.5 GHz). The SKA1 will produce a raw data rate of ~10 Tb/s, require a computing power of 100 Pflop/s and an archiving capacity of hundreds of PB/year. The next phase of the project, the SKA2, is going to increase the number of both low and mid antennas by a factor of 10 and increase the computing requirements accordingly. The key requirements for the project are a very demanding availability of 99.9%, computing scalability and result reproducibility. We propose an approach to enforce these requirements - with an optimal use of resources - by using highly distributed computing and virtualization technologies.publishe

TM Services: an architecture for monitoring and controlling the Square Kilometre Array (SKA) Telescope Manager (TM)

The SKA project is an international effort (10 member and 10 associated countries with the involvement of 100 companies and research institutions) to build the world's largest radio telescope. The SKA Telescope Manager (TM) is the core package of the SKA Telescope aimed at scheduling observations, controlling their execution, monitoring the telescope and so on. To do that, TM directly interfaces with the Local Monitoring and Control systems (LMCs) of the other SKA Elements (for example, Dishes, Correlator and so on), exchanging commands and data with them by using the TANGO controls framework (see [1]). TM in turn needs to be monitored and controlled, in order its continuous and proper operation is ensured and this higher responsibility has been assigned to the TM SER package

A Cyber Infrastructure for the SKA Telescope Manager

The Square Kilometre Array Telescope Manager (SKA TM) will be responsible for assisting the SKA Operations and Observation Management, carrying out System diagnosis and collecting Monitoring & Control data from the SKA subsystems and components. To provide adequate compute resources, scalability, operation continuity and high availability, as well as strict Quality of Service, the TM cyber-infrastructure (embodied in the Local Infrastructure - LINFRA) consists of COTS hardware and infrastructural software (for example: server monitoring software, host operating system, virtualization software, device firmware), providing a specially tailored Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) solution. The TM infrastructure provides services in the form of computational power, software defined networking, power, storage abstractions, and high level, state of the art IaaS and PaaS management interfaces. This cyber platform will be tailored to each of the two SKA Phase 1 telescopes (SKA_MID in South Africa and SKA_LOW in Australia) instances, each presenting different computational and storage infrastructures and conditioned by location. This cyber platform will provide a compute model enabling TM to manage the deployment and execution of its multiple components (observation scheduler, proposal submission tools, M&C components, Forensic tools and several Databases, etc). In this sense, the TM LINFRA is primarily focused towards the provision of isolated instances, mostly resorting to virtualization technologies, while defaulting to bare hardware if specifically required due to performance, security, availability, or other requirement

Experimental evaluation of the usage of ad hoc networks as stubs for multiservice networks

This paper describes an experimental evaluation of a multiservice ad hoc network, aimed to be interconnected with an infrastructure, operator-managed network. This network supports the efficient delivery of services, unicast and multicast, legacy and multimedia, to users connected in the ad hoc network. It contains the following functionalities: routing and delivery of unicast and multicast services; distributed QoS mechanisms to support service differentiation and resource control responsive to node mobility; security, charging, and rewarding mechanisms to ensure the correct behaviour of the users in the ad hoc network. This paper experimentally evaluates the performance of multiple mechanisms, and the influence and performance penalty introduced in the network, with the incremental inclusion of new functionalities. The performance results obtained in the different real scenarios may question the real usage of ad-hoc networks for more than a minimal number of hops with such a large number of functionalities deployed

The CPLEAR detector at CERN

The CPLEAR collaboration has constructed a detector at CERN for an extensive programme of CP-, T- and CPT-symmetry studies using ${\rm K}^0$ and $\bar{\rm K}^0$ produced by the annihilation of $\bar{\rm p}$'s in a hydrogen gas target. The ${\rm K}^0$ and $\bar{\rm K}^0$ are identified by their companion products of the annihilation ${\rm K}^{\pm} \pi^{\mp}$ which are tracked with multiwire proportional chambers, drift chambers and streamer tubes. Particle identification is carried out with a liquid Cherenkov detector for fast separation of pions and kaons and with scintillators which allow the measurement of time of flight and energy loss. Photons are measured with a lead/gas sampling electromagnetic calorimeter. The required antiproton annihilation modes are selected by fast online processors using the tracking chamber and particle identification information. All the detectors are mounted in a 0.44 T uniform field of an axial solenoid of diameter 2 m and length 3.6 m to form a magnetic spectrometer capable of full on-line reconstruction and selection of events. The design, operating parameters and performance of the sub-detectors are described.